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The  main  results  obtained  and  published  during  the  period  covered 
by  this  report,  June  1984  -  May  1988,  are  described  below  together 
with  references  given  to  the  corresponding  publication. 

1.  Dual  Control  Guidance  for  Simultaneous  Identification  and 

Interception 

(by  K.  Birmiwal  and  Y.  Bar-Shalom  in  Automatica.  Vol.  20,  No.  6,  pp. 
737-749,  NOV.  1984) 

An  adaptive  dual-control  guidance  algorithm  was  developed  for 
intercepting  a  moving  target  in  the  presence  of  an  interfering  target 
(decoy)  in  a  stochastic  environment.  Two  sequences  of  measurements 
are  obtained  at  discrete  points  in  time;  however,  it  is  not  certain 
which  sequence  came  from  the  target  of  interest  and  which  from  the 
decoy.  Associated  with  each  track,  the  interceptor  also  receives 
noisy,  state-dependent  feature  measurements.  The  optimum  control  for 
the  interceptor,  which  is  given  by  the  solution  of  the  stochastic 
dynamic  programming  equation,  is  not  numerically  feasible  to  obtain. 
An  approximate  solution  of  this  equation  is  obtained  by  evaluating 
the  value  of  the  future  information  gathering.  This  is  done  through 
the  use  of  preposterior  analysis:  approximate  prior  probability 
densities  are  obtained  and  used  to  describe  the  future  learning  and 
control.  In  this  way,  the  interceptor  control  is  used  for 

information  gathering  in  order  to  reduce  the  future  target  and  decoy 
inertial  measurement  errors  and  enhance  the  observable  target/decoy 
feature  differences  for  subsequent  discrimination  between  the  true 
target  and  the  decoy.  Simulation  studies  have  shown  the 

effectiveness  of  the  scheme. 
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2.  Dual  Adaptive  Control  Based  Upon  Sensitivity  Functions 

(by  J.A.  Molusis,  P.  Mookerjee,  Y.  Bar-Shalom,  in  Proc.  23rd  IEEE 
Conference  on  Decision  &  Control,  Dec.  1984) 

A  new  adaptive  dual  control  solution  was  developed  for  the 
control  of  a  class  of  multi-variable  input-output  systems.  Both 
rapidly  varying  random  parameters  and  constant  but  unknown  parameters 
are  included.  The  new  controller  modifies  the  cautious  control 
design  by  numerator  and  denominator  correction  terms.  This 
controller  is  shown  to  depend  upon  sensitivity  functions  of  the 
expected  future  cost.  A  scalar  example  is  presented  to  provide 
insight  into  the  properties  of  the  new  dual  controller.  Monte-Carlo 
simulations  are  performed  which  show  improvement  over  the  cautious 
controller  and  the  previous  Linear  Feedback  Dual  Controller. 

3.  Dual  Control  and  Prevention  of  the  Turn-Off  Phenomenon  in  a  Class 
of  MIMO  Systems 

(by  P.  Mookerjee,  Y.  Bar-Shalom,  and  J.A.  Molusis,  in  Proc.  24th  IEEE 
Conference  on  Decision  &  Control.  Dec.  1985) 

The  recently  developed  methodology  of  adaptive  dual  control 
bassed  upon  sensitivity  functions  was  applied  here  to  a 
multi-variable  input-output  model.  The  plant  has  constant  but 
unknown  parameters.  It  represents  a  simplified  linear  version  of  the 
relationship  between  the  vibration  output  and  the  higher  harmonic 
control  input  for  a  helicopter.  The  cautious  and  the  new  dual 
controller  are  examined.  In  many  instances,  the  cautious  controller 
is  seen  to  turn  off.  The  new  dual  controller  modifies  the  cautious 
control  design  by  numerator  and  denominator  correction  terms  which 
depend  upon  the  sensitivity  functions  of  the  expected  future  cost  and 
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avoids  the  turn-off  and  burst  phenomena.  Monte-Carlo  simulations  and 
statistical  tests  of  significance  indicate  the  superiority  of  the 
dual  controller  over  the  cautious  and  the  heuristic  certainty 
equivalence  controllers. 

4.  An  Adaptive  Dual  Controller  for  a  MIMO-ARMA  System 

(by  P.  Mookerjee  and  Y.  Bar-Shalom,  in  Proc.  1986  IEEE  Conference  on 
.  Decision  &  Control,  to  appear  in  IEEE  Trans.  Auto.  Control.  April 
1989) 

An  adaptive  dual  controller  was  developed  for  a  multi- input 
multi-output  dynamic  system.  The  plant  has  constant  but  unknown 
parameters.  The  cautious  and  a  new  dual  controller  are  examined.  In 
many  instances,  the  cautious  controller  is  seen  to  turn  off.  The 
dual  controller  modifies  the  cautious  control  design  by  numerator  and 
denominator  correction  terms  which  depend  upon  the  sensitivity 
functions  of  the  expected  future  cost  and  avoids  the  turn-off  and 
slow  convergence.  Monte-Carlo  comparisons  based  on  parametric  and 
nonparametric  statistical  analysis  indicate  the  superiority  of  the 
dual  controller  over  the  cautious  controller. 

5.  Sojourn  Time  Distribution  in  a  Class  of  Semi-Markov  Chains 

(by  P.  Mookerjee,  L.  Campo,  and  Y.  Bar-Shalom,  in  Proc.  1987  Conf.  on 
Information  Sciences  and  Systems.  Johns  Hopkins  University, 
Baltimore,  MD,  March  1987. 

The  estimation  and  control  of  a  plant  with  stochastic  parameters 
subject  to  random  input  disturbances  and  measurement  errors  can  be 
handled  by  adopting  the  usual  techniques  of  state/parameter 
estimation  and  choosing  an  appropriate  control  policy,  when  the 

parameters  are  constant  or  slowly  varying  with  time.  Not  all 
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practical  processes  can,  however,  be  solved  as  above.  There 
arenumerous  examples  of  systems  which  have  discrete  models  that 
randomly  vary  with  time  and  experience  switchings  between  different 
models  after  a  random  sojourn  time.  In  some  situations  the  switching 
probabilities  depend  on  the  sojourn  time.  Such  instances  are 
commonly  experienced  in  target  tracking,  manufacturing  systems,  power 
industry  and  also  in  the  socio-economic  environment.  In  such  a 
system,  knowledge  of  the  sojourn  time  (in  the  form  of  a  conditional 
distribution)  is  needed  to  infer  the  transition  probabilities.  In 
this  paper  we  derive  the  conditional  sojourn  time  distribution  for 
systems  with  imperfect  observations  and  changing  structures  of 
models. 

6.  State  Estimation  for  Systems  with  Sojourn-Time-Dependent  Markov 

Model-Switching 

(by  L.  Campo,  P.  Mookerjee,  and  Y.  Bar-Shalom,  in  Proc.  IEEE  Conf. 
on  Decision  and  Control.  Dec.  1987;  to  appear  in  IEEE  Trans.  Auto. 
Control) 

There  are  numerous  examples  of  systems  which  have  discrete  models 
that  randomly  vary  with  time  and  experience  switchings  between 
different  models  after  a  random  sojourn  time.  In  some  situations  the 
switching  probabilities  depend  on  the  sojourn  time.  Such  a  switching 
process,  discussed  in  this  paper,  is  a  class  of  semi-Markov  processes 
and  is  encountered  in  target  tracking,  systems  subject  to  failures, 
and  also  in  the  socio-economic  environment.  In  such  a  system,  the 
transition  probabilities  depend  on  the  sojourn  time.  It  is  shown  in 
this  paper  how  one  can  infer  the  transition  probabilities  via  the 
evaluation  of  the  conditional  distribution  of  the  sojourn  time. 
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Subsequently,  using  the  conditional  sojourn  time  distribution  for 
dynamic  systems  with  imperfect  observations  and  changing  structures 
or  models,  a  recursive  state  estimation  for  such  systems  is 
presented . 

7.  The  Interacting  Multiple  Model  Algorithm  for  Systems  with 
Markovian  Coefficients 

(by  H.A.P.  Blom  and  Y.  Bar-Shalom,  IEEE  Trans.  Auto.  Control.  Aug. 
1988) 

A  major  problem  in  filtering  for  linear  systems  with  Markovian 
switching  coefficients  is  the  problem  of  hypotheses  management.  For 
this  dynamic  multiple  model  situation  the  paper  presents  a  novel 
approach  to  hypotheses  merging.  The  novelty  lies  in  an  alternative 
timing  of  hypotheses  merging.  Applied  to  the  problem  of  filtering  in 
a  linear  system  with  Markovian  coefficients  it  yields  an  elegant  way 
to  derive  the  IMM  (Interacting  Multiple  Model)  algorithm.  After  its 
derivation  follows  a  further  evaluation  of  the  IMM  algorithm  and  a 
comparison  with  other  algorithms.  This  makes  clear  that  the  IMM 
performs  very  well  at  a  relatively  low  computational  load.  These 
results  imply  a  significant  change  to  the  picture  of  suboptimal 
Bayesian  filtering  for  systems  with  Markovian  coefficients. 

8.  From  Piecewise  Deterministic  to  Piecewise  Diffusion  Markov 
Processes 

(by.  H.A.P.  Blom,  in  Proc.  IEEE  conf.  on  Decision  and  Control.  Dec. 
1988) 

Piecewise  Deterministic  (PD)  Markov  Processes  form  a  remarkable 
class  of  hybrid  state  processes  because,  in  contrast  to  roost  other 
hybrid  state  processes,  they  include  a  jump  reflecting  boundary  and 


exclude  diffusion.  As  such,  they  cover  a  wide  variety  of  impulsively 


or  singularly  controlled  non-diffusion  processes.  Because  PD 
processes  are  defined  in  a  pathwise  way,  they  provide  a  framework  to 
study  the  control  of  non-diffusion  processes  along  the  same  lines  as 
that  of  diffusions.  An  important  generalization  is  to  include 
diffusion  in  PD  processes,  but,  as  pointed  out  by  Davis,  combining 
diffusion  with  a  jump  reflecting  boundary  seems  not  possible  within 
the  present  definition  of  PD  processes.  This  paper  presents  PD 
processes  as  pathwise  unique  solutions  of  an  IT  stochastic 
differential  equation  (SDE) ,  driven  by  a  Poisson  random  measure. 
Since  such  an  SDE  permits  the  inclusion  of  diffusion,  this  approach 
leads  to  a  large  variety  of  piecewise  diffusion  Markov  processes, 
represented  by  pathwise  unique  SDE  solutions. 

9.  Failure  Detection  via  Recursive  Estimation  for  a  Class  of 

Semi-Markov  Switching  Systems 

(by  L.  Campo,  P.  Mookerjee,  and  Y.  Bar-Shalom,  Proc.  IEEE  Conf.  on 
Decision  and  Control.  Dec.  1988) 

Systems  with  parameters  that  change  in  value  due  to  component 
failures  can  be  represented  by  a  discrete  set  of  models.  For  systems 
where  the  model  switching  process  is  Semi-Markov,  a  Semi-Markov  based 
estimation  scheme  is  required  to  obtain  the  optimal  state  estimate. 
The  "sojourn-time-dependent  Markov"  (STDM)  model  switching  is  of  the 
Semi-Markov  type.  Our  recently  developed  STDM-based  multiple-model 
recursive  estimator  computes  conditional  model  probabilities  which 
can  be  used  to  determine  when  a  failure  model  is  in  effect.  We  show 
through  an  example  that  this  estimator  is  better  able  to  determine 
which  model  is  in  effect  than  the  Markov-based  multiple-model 
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algorithm  and  thus  the  STDM-based  algorithm  is  a  more  reliable 
failure  detection  scheme.  Our  example  also  shows  that  the  STDM-based 
algorithm  gives  a  better  state  estimate,  in  terms  of  the  rms  error, 
than  a  Markov-based  scheme. 

10.  Time-Reversion  of  a  Hybrid  State  Stochastic  Difference  System 

with  Jump-Linear  Smoothing  Application 
(by  H.A.P.  Blom  and  Y.  Bar-Shalom,  submitted  to  IEEE  International 
Conf .  on  Control  and  Applications.  April  1989  and  IEEE  Trans.  Auto. 
Control) 

The  reversion  in  time  of  a  stochastic  difference  equation  in  a 
hybrid  space,  with  a  Markovian  solution,  is  presented.  The 
reveresion  is  obtained  by  a  martingale  approach,  which  previously  led 
to  reverse  time  forms  for  stochastic  equations  with  Gauss-Markov  or 
diffusion  solutions.  The  reverse  time  equations  follow  from  a 
particular  non-canonical  martingale  decomposition,  while  the  reverse 
time  equations  for  Gauss-Markov  and  diffusion  solutions  followed  from 
the  canonical  martingale  decomposition.  The  need  for  this 
non-canonical  decomposition  stems  from  the  hybrid  state  space 
situation.  Moreover,  the  non-Gaussian  discrete  time  situation  leads 
to  reverse  time  equations  that  incorporate  a  Bayesian  estimation 
step.  The  latter  step  is  carried  out  for  linear  systems  with 
Markovian  switching  coefficients,  and  the  result  is  shown  to  provide 
the  solution  to  the  problem  of  fixed-interval  smoothing.  For  an 
application  of  this  smoothing  approach  to  a  trajectory  wit!  sudden 
maneuvers,  simualtion  results  are  given  to  illustrate  the  practical 
use  of  the  reverse  time  equations  obtained. 


